Image processing method, image processing apparatus, robot-mounted transfer device, and system

ABSTRACT

A system includes a machine tool  10 , a robot  25  having a camera  31 , and a transportation device  35  having the robot  25  mounted thereon, and an identification figure is arranged in a machining area of the machine tool  10.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present disclosure relates to a machine tool which machines aworkpiece, and a robot-mounted transportation device which includes arobot performing an operation with respect to the machine tool and atransportation device moving with the robot mounted thereon, and alsorelates to a system which includes the machine tool and therobot-mounted transportation device. The present disclosure furtherrelates to an image processing method and an image processing apparatus.

2. Description of the Related Art

A known example of a system as mentioned above is disclosed in JapaneseUnexamined Patent Application Publication No. 2017-132002. This systemis configured such that an automatic guided vehicle (AGV) having a robotmounted thereon moves to an operation position set with respect to amachine tool, and the robot performs operations, such as attachment andremoval of a workpiece, with respect to the machine tool at theoperation position.

Such a system enables a single robot which is moved by an automaticguided vehicle to perform operations, such as attachment and removal ofa workpiece, with respect to more than one machine tool. Therefore, ascompared with a system in which a robot is arranged in a fixed mannerwith respect to a machine tool, the degree of freedom in machine toollayout is increased so that a machine tool layout which providesenhanced production efficiency is possible. Further, since it ispossible to cause a single robot to perform operations with respect tomany machine tools, equipment costs are reduced as compared with theconventional system in which the robot is arranged in a fixed manner.

However, because the automatic guided vehicle is configured to moveitself by means of wheels, the automatic guided vehicle cannot always bestopped at the operation position with high positioning accuracy.Therefore, in order that the robot accurately performs operations withrespect to the machine tool, it is necessary to, when the automaticguided vehicle is positioned at the operation position, compare the poseof the robot with a reference pose of the robot, which is set inso-called teaching and serves as a reference for control, detect anamount of error between them, and compensate an operating pose of therobot based on the detected amount of error.

A known technique for such robot-pose compensation is disclosed as aposition compensation method in Japanese Unexamined Patent ApplicationPublication No. 2016-221622. Specifically, this position compensationmethod is configured such that a visual target consisting of twocalibration markers is arranged on an outer surface of the machine tool,images of the visual target are captured by a camera arranged on amovable part of the robot, a relative positional relation between therobot and the machine tool is measured based on the captured images andthe position and pose of the camera, and an operating pose of the robotis compensated based on the measured positional relation.

However, in the above-described conventional position compensationmethod, for example, when a hand or the like of the robot is inserted inthe machine tool to cause the hand to attach or remove a workpiece to orfrom a chuck or the like of the machine tool, the pose of the robot forperforming the attachment or removal is not accurately compensated.

Specifically, because the automatic guided vehicle is configured to bemoved by operation of the wheels that have a relatively high degree offreedom, the automatic guided vehicle has the characteristics that therobot-mounted surface is easily tilted toward the floor and that thetilt of the robot-mounted surface easily varies due to change of thepose of the robot mounted thereon, in other words, due to change of theposition of the center of gravity of the robot.

Therefore, when the robot is in a pose having the hand thereof insertedin the machine tool to attach or remove a workpiece, in other words,when an arm of the robot is overhanging to a great extent from theautomatic guided vehicle, the tilt of the robot-mounted surface isgreater than that when the hand of the robot is positioned outside themachine tool and the arm is not overhanging from the automatic guidedvehicle or is overhanging only to a very slight extent.

Therefore, where, as in the above-described conventional positioncompensation method, a visual target as a calibration marker is arrangedon an outer surface of the machine tool and an amount of positioncompensation (amount of pose compensation) for the robot is obtainedwith the robot positioned outside the machine tool, the pose of therobot for attachment or removal of a workpiece that is performed withthe hand of the robot positioned inside the machine tool cannot beaccurately compensated based on the obtained amount of positioncompensation.

Where the pose of the robot for attachment or removal of a workpiececannot be accurately compensated, the hand of the robot cannot beaccurately positioned with respect to the chuck. For example, in thecase where the chuck is such that its clamping part has only a verysmall movement allowance (stroke), i.e., there is only a very smallclearance between the workpiece and the chuck, such as in the case of acollet chuck, the chuck may fail to reliably clamp the workpiece.

Further, where attachment or removal of a workpiece is not reliablycarried out, availability of the system is reduced.

SUMMARY OF THE INVENTION

Accordingly, the present disclosure provides a system, a transportationdevice, an image processing method, and an image processing apparatus asset forth in the appended claims.

In the present disclosure, an operating pose of the robot is compensatedbased on an image including an identification figure, so that theoperating pose of the robot is compensated more accurately.

Further, in the present disclosure, an operating pose of the robot iscompensated based on an image including an internal structure of themachine tool, so that the operating pose of the robot is compensatedmore accurately.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view schematically illustrating a structure of a systemaccording to an embodiment of the present invention;

FIG. 2 is a block diagram illustrating a configuration of the systemaccording to the embodiment;

FIG. 3 is a perspective view illustrating an automatic guided vehicleand a robot in the embodiment;

FIG. 4 is a diagram for explaining an image capturing pose of the robotin the embodiment;

FIG. 5 is a diagram illustrating a relation between coordinate systemsset in the embodiment;

FIG. 6 is a diagram illustrating an identification figure in theembodiment;

FIG. 7 is a diagram illustrating an operating pose of a hand compensatedbased on an image of the identification figure captured in actualoperation in the case where there is no error between an image capturingpose of a camera in teaching operation and an image capturing pose ofthe camera in the actual operation;

FIG. 8 is a diagram illustrating an image of the identification figurecaptured by the camera in the teaching operation;

FIG. 9 is a diagram illustrating the case where there is an errorbetween the image capturing pose of the camera in the teaching operationand the image capturing pose of the camera in the actual operation;

FIG. 10 is a diagram illustrating images of the identification figurecaptured by the camera as being in the image capturing pose in theteaching operation and as being in the image capturing pose in theactual operation in the case where there is an error between the imagecapturing pose of the camera in the teaching operation and the imagecapturing pose of the camera in the actual operation;

FIG. 11 is a diagram illustrating an operating pose of the handcompensated based on the image of the identification figure captured inthe actual operation in the case where there is an error between theimage capturing pose of the camera in the teaching operation and theimage capturing pose of the camera in the actual operation;

FIG. 12 is a diagram illustrating a manner of compensation for makingthe image capturing pose of the camera in the actual operation closer tothe image capturing pose of the camera in the teaching operation in thecase where there is an error between the image capturing pose of thecamera in the teaching operation and the image capturing pose of thecamera in the actual operation;

FIG. 13 is a diagram illustrating a state where the operating pose ofthe hand is compensated based on an image of the identification figurecaptured with the image capturing pose of the camera in the actualoperation compensated;

FIG. 14 is a flowchart illustrating how the automatic guided vehicle andthe robot are operated under automatic operation control in thisembodiment;

FIG. 15 is a diagram illustrating a variation of the configuration withthe identification figure arranged in the machine tool; and

FIG. 16 is a diagram illustrating another variation of the configurationwith the identification figure arranged in the machine tool.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, a specific embodiment of the present invention will bedescribed with reference to the drawings.

[Configuration of System According to this Embodiment]

As illustrated in FIGS. 1 and 2, a system 1 according to this embodimentincludes a machine tool 10, a material storage 20 and a product storage21 as peripheral devices, an automatic guided vehicle (AGV) 35, a robot25 mounted on the automatic guided vehicle 35, a camera 31 attached tothe robot 25, and a controller 40 controlling the robot 25 and theautomatic guided vehicle 35. Note that the robot 25, the automaticguided vehicle 35, and the controller 40 constitute a robot-mountedtransportation device.

As illustrated in FIG. 4, the machine tool 10 is an NC(numerically-controlled) vertical lathe having a spindle 11 arrangedvertically to which a chuck 12 for clamping a workpiece W (W) isattached. The machine tool 10 is capable of performing turning on aworkpiece W (W′). The machine tool 10 has a tool presetter 13 arrangedin the vicinity of the spindle 11. The tool presetter 13 has a contactor14 and a support bar 15 supporting the contactor 14. The support bar 15is movable into and out of a machining area along an axis of the spindle11 and has a display board 16, which is made of, for example but notlimited to, ceramics, provided on an end surface thereof located on themachining area side. The display board 16 has an identification figure,as illustrated in FIG. 6, drawn thereon. Note that the display board 16is arranged to be on a horizontal plane. Thus, the identification figurein this embodiment is disposed in the machine tool 10 and it is arrangedin the machining area, which is a particularly preferable configuration.

FIG. 4 shows a state where the support bar 15 and the contactor 14 havebeen moved into the machining area. After the support bar 15 and thecontactor 14 are moved out of the machining area so that the contactor14 and the display board 16 are retracted into a storage space, ashutter 17 is closed so that the contactor 14 and the display board 16are isolated from the machining area.

The identification figure in this example has a matrix structure havinga plurality of square pixels arranged two-dimensionally, each pixelbeing displayed in white or black. In FIG. 6, the pixels displayed inblack are hatched. Examples of the identification figure includeso-called “AR marker” and “AprilTag”. Further, where a smallidentification figure is used, some measure, e.g., arranging a lens overthe identification figure, may be taken to enable the camera 31, whichis described later, to capture an enlarged image of the identificationfigure.

The material storage 20 is disposed on the left of the machine tool 10in FIG. 1. The material storage 20 stores therein materials (unmachinedworkpieces W) to be machined in the machine tool 10. The product storage21 is disposed on the right of the machine tool 10 in FIG. 1. Theproduct storage 21 stores therein products or semi-finished products(machined workpieces W) machined in the machine tool 10.

As illustrated in FIG. 1, the automatic guided vehicle 35 has a mountsurface 36 as its top surface, on which the robot 25 is mounted.Further, the automatic guided vehicle 35 has an operation panel 37attached thereto which an operator can carry around. The operation panel37 has an input and output unit for input and output of data, anoperation unit for manual operation of the automatic guided vehicle 35and the robot 25, and a display capable of displaying a picture thereon.

Further, the automatic guided vehicle 35 has a sensor (for example, adistance measurement sensor using a laser beam) which enablesrecognition of the position of the automatic guided vehicle 35 in aplant, and the automatic guided vehicle 35 is configured to traveltracklessly in the plant, including the area where the machine tool 10,the material storage 20, and the product storage 21 are disposed, undercontrol by the controller 40. The automatic guided vehicle 35 in thisembodiment moves to operation positions respectively set with respect tothe machine tool 10, the material storage 20, and the product storage21.

As illustrated in FIGS. 1 and 3, the robot 25 in this embodiment is anarticulated robot having three arms, namely, a first arm 26, a secondarm 27, and a third arm 28. The third arm 28 has a hand 29 as an endeffector attached to a distal end thereof, and also has one camera 31attached to the distal end thereof via a support bar 30. Note that therobot applicable in the present invention is not limited to thisconfiguration. The robot needs only to have (i) a camera, (ii) a handunit for gripping a workpiece or a tool, (iii) a second arm unit towhich the hand unit is movably connected, and (iv) a first arm unit towhich the second arm unit is movably connected. As compared with therobot 25 in this embodiment, the hand 29 corresponds to the hand unit,the second arm 27 and a joint rotatably (movably) coupled to the secondarm 27 correspond to the second arm unit, and the first arm 26 and ajoint rotatably (movably) coupled to the first arm 26 correspond to thefirst arm unit. Note that the third arm 28 and a joint rotatably andreciprocatably (movably) coupled to the third arm 28 may be deemed tocorrespond to the second arm unit. That is to say, although the robot inthis embodiment has three arms, the robot needs only to have at leasttwo arms.

As illustrated in FIG. 2, the controller 40 in this embodiment consistsof an operation program storage 41, a moving position storage 42, anoperating pose storage 43, a map information storage 44, anidentification figure image storage 45, a manual-operation control unit46, an automatic-operation control unit 47, a map information generator48, a position recognition unit 49, and an input and output interface50. The controller 40 is connected to the machine tool 10, the materialstorage 20, the product storage 21, the robot 25, the camera 31, theautomatic guided vehicle 35, and the operation panel 37 via the inputand output interface 50. Note that the controller 40 is not limited tothis configuration. The controller 40 needs only to include at least afunctional unit for controlling the robot 25, and other functional unitsmay be included in other devices.

Note that the controller 40 is composed of a computer including a CPU, aRAM, and a ROM. The manual-operation control unit 46, theautomatic-operation control unit 47, the map information generator 48,the position recognition unit 49, and the input and output interface 50are functionally implemented by a computer program to carry out theprocesses described later. The operation program storage 41, the movingposition storage 42, the operating pose storage 43, the map informationstorage 44, and the identification figure image storage 45 are composedof an appropriate storage medium, such as a RAM. In this embodiment, thecontroller 40 is attached to the automatic guided vehicle 35, and isconnected to the machine tool 10, the material storage 20, and theproduct storage 21 through appropriate communication means and connectedto the robot 25, the camera 31, the automatic guided vehicle 35, and theoperation panel 37 by wire or wirelessly. However, the controller 40 isnot limited to this configuration and may be disposed at an appropriateposition other than the automatic guided vehicle 35. In such a case, thecontroller 40 is connected to the above-mentioned elements throughappropriate communication means.

The manual-operation control unit 46 is a functional unit that operatesthe automatic guided vehicle 35, the robot 25, and the camera 31 inaccordance with operation signals input through the operation panel 37by an operator. That is to say, an operator can manually operate theautomatic guided vehicle 35, the robot 25, and the camera 31 through theoperation panel 37, which is controlled by the manual-operation controlunit 46.

The operation program storage 41 is a functional unit that storestherein an automatic-operation program for causing the automatic guidedvehicle 35 and the robot 25 to automatically operate during production,and a map generation program for causing the automatic guided vehicle 35to operate during generation of map information of the plant, which isdescribed later. The automatic-operation program and the map generationprogram are stored into the operation program storage 41, for example,by being input through the input and output unit of the operation panel37.

The automatic-operation program contains command codes regarding amoving position as a target position to which the automatic guidedvehicle 35 is moved, a moving speed of the automatic guided vehicle 35,and an orientation of the automatic guided vehicle 35. Theautomatic-operation program further contains command codes regardingoperations to be carried out in sequence by the robot 25 and commandcodes for causing the camera 31 to operate. The map generation programcontains command codes for causing the automatic guided vehicle 35 totravel tracklessly all over the plant to cause the map informationgenerator 48 to generate map information.

The map information storage 44 is a functional unit that stores thereinmap information including information on arrangement of machines,devices, instruments, etc. (hereinafter, collectively referred to as“devices”) arranged in the plant where the automatic guided vehicle 35travels. The map information is generated by the map informationgenerator 48.

The map information generator 48 obtains spatial information of theplant from distance data detected by the sensor when the automaticguided vehicle 35 is caused to travel in accordance with the mapgeneration program stored in the operation program storage 41 undercontrol by the automatic-operation control unit 47, which is describedin detail later, of the controller 40. The map information generator 48also recognizes planar shapes of the devices arranged in the plant, and,for example, based on previously registered planar shapes of thedevices, recognizes the positions, planar shapes, etc. of particulardevices (in this example, the machine tool 10, the material storage 20,and the product storage 21) arranged in the plant (arrangementinformation). The map information generator 48 stores the obtainedspatial information and arrangement information as map information ofthe plant into the map information storage 44.

The position recognition unit 49 is a functional unit that recognizesthe position of the automatic guided vehicle 35 in the plant based ondistance data detected by the sensor and the map information of theplant stored in the map information storage 44. Based on the position ofthe automatic guided vehicle 35 recognized by the position recognitionunit 49, the automatic-operation control unit 47 controls operation ofthe automatic guided vehicle 35.

The moving position storage 42 is a functional unit that stores thereinspecific moving positions. The moving positions are specific targetpositions to which the automatic guided vehicle 35 is moved, andcorrespond to the above-described command codes contained in theoperation programs. The moving positions include the above-mentionedoperation positions set with respect to the machine tool 10, thematerial storage 20, and the product storage 21. Note that the movingpositions are set, for example, as follows: the automatic guided vehicle35 is manually operated through the operation panel 37 so that it ismoved to each targeted position under control by the manual-operationcontrol unit 46, and position data recognized by the positionrecognition unit 49 at each targeted position is stored into the movingposition storage 42. This operation is generally called “teachingoperation”.

The operating pose storage 43 is a functional unit that stores thereindata regarding poses (operating poses) of the robot 25, into which therobot 25 is brought in sequence when it is operated in a predeterminedsequence. The operating poses correspond to the command codes containedin the operation program. This operating pose data is composed ofrotational angle data of joints (motors) of the robot 25 in eachtargeted pose. This rotational angle data is obtained by, in theteaching operation using the operation panel 37, manually operating therobot 25 to bring the robot 25 into each targeted pose under control bythe manual-operation control unit 46. The obtained rotational angle datais stored as operating pose data into the operating pose storage 43.

Specific operating poses of the robot 25 are set with respect to each ofthe material storage 20, machine tool 10, and product storage 21. Forexample, a set of extraction poses is set with respect to the materialstorage 20, the set of extraction poses consisting of an operationstarting pose (extraction starting pose) for starting extraction withrespect to the material storage 20, operating poses (extracting poses)for causing the hand 29 to grip an unmachined workpiece W stored in thematerial storage 20 and extract the unmachined workpiece W from thematerial storage 20, and a pose for finishing the extraction (extractionfinishing pose; in this embodiment, this pose is identical to theextraction starting pose).

A set of workpiece-removal poses for removing a machined workpiece Wfrom the machine tool 10 and a set of workpiece-attachment poses forattaching an unmachined workpiece W to the machine tool 10 are set withrespect to the machine tool 10.

Specifically, the set of workpiece-removal poses consists of, forexample, an operation starting pose preceding insertion into the machinetool 10, a pose for moving the hand 29 and the camera 31 into themachining area of the machine tool 10 and casing the camera 31 tocapture an image of the identification figure provided on the supportbar 15 (image capturing pose; see FIG. 4), a pose for positioning thehand 29 opposite a machined workpiece W′ clamped by the chuck 12 of themachine tool 10 (removal preparing pose), a pose for moving the hand 29toward the chuck 12 and causing the hand 29 to grip the machinedworkpiece W′ clamped by the chuck 12 (gripping pose), a pose for movingthe hand 29 away from the chuck 12 to pull the machined workpiece W′from the chuck 12 (pulling pose), and a pose for moving the hand 29 andthe camera 31 out of the machine tool 10 (operation finishing pose).

The set of workpiece-attachment poses consists of, for example, anoperation starting pose preceding insertion into the machine tool 10, apose for moving the hand 29 and the camera 31 into the machining area ofthe machine tool 10 and causing the camera 31 to capture an image of theidentification figure provided on the support bar 15 (image capturingpose; see FIG. 4), a pose for positioning a unmachined workpiece Wgripped by the hand 29 opposite the chuck 12 of the machine tool 10(attachment preparing pose), a pose for moving the hand 29 toward thechuck 12 to allow the chuck 12 to clamp the unmachined workpiece W(attaching pose), a pose for moving the hand 29 away from the chuck 12(moving-away pose), and a pose for moving the hand 29 and the camera 31out of the machine tool 10 (operation finishing pose).

A set of storage poses is set with respect to the product storage 21,the set of storage poses consisting of an operation starting pose forstarting storage with respect to the product storage 21 (storagestarting pose), operating poses for storing a machined workpiece W′gripped by the hand 29 into the product storage 21 (storing poses), anda pose for finishing the storage (storage finishing pose; in thisembodiment, this pose is identical to the storage starting pose).

The identification figure image storage 45 is a functional unit thatstores therein images of the identification figure provided on thesupport bar 15 of the tool presetter 13 captured by the camera 31 whenthe automatic guided vehicle 35 is at the operation position set withrespect to the machine tool 10 and the robot 25 is in the imagecapturing pose in the teaching operation and in automatic operation.Note that the image of the identification figure captured in theteaching operation is stored as a reference image in the identificationfigure image storage 45. Note further that an image capturing poseposition, an attaching pose position, and other positions in a figurecoordinate system are stored in an appropriate storage.

The automatic-operation control unit 47 is a functional unit thatoperate the automatic guided vehicle 35, the robot 25, and the camera 31in accordance with the automatic-operation program or map generationprogram stored in the operation program storage 41. In this process, thedata stored in the moving position storage 42 and the operating posestorage 43 are used as necessary.

Unmanned and Automated Production in System According to this Embodiment

In the system 1 according to this embodiment, unmanned and automatedproduction is performed in the following manner under control by theautomatic-operation control unit 47.

That is to say, the automatic-operation program stored in the operationprogram storage 41 is executed by the automatic-operation control unit47, so that, for example, the automatic guided vehicle 35 and the robot25 operate in the following manner.

First, the automatic guided vehicle 35 is moved to the operationposition set with respect to the machine tool 10 and the robot 25 isbrought into the operation starting pose for the above-describedworkpiece removal. At this time, the machine tool 10 has finished apredetermined machining operation, a door cover thereof has been openedso that the robot 25 can enter the machining area, and the support bar15 of the tool presetter 13 has been moved into the machining area uponreceipt of a command from the automatic-operation control unit 47.

Subsequently, the automatic-operation control unit 47 causes the robot25 to carry out the rest of the workpiece removal, so that a machinedworkpiece W′ clamped by the chuck 12 of the machine tool 10 is grippedby the hand 29 and removed from the machine tool 10. In this process,the automatic-operation control unit 47 causes the camera 31 to capturean image of the identification figure when the robot 25 is in the imagecapturing pose, and compensates the subsequent removal preparing pose,gripping pose, and pulling pose based on the captured image. Asmentioned in the foregoing, the automatic guided vehicle 35 stops at theoperation position with low positioning accuracy since the automaticguided vehicle 35 is configured to move itself by means of wheels.Therefore, in actual operation, it is necessary to compensate theoperating poses set in advance in the teaching operation. A specificmanner of this compensation is described later. Note that, afterbringing the robot 25 into the gripping pose, the automatic-operationcontrol unit 47 transmits a chuck open command to the machine tool 10 toopen the chuck 12.

Subsequently, the automatic-operation control unit 47 moves theautomatic guided vehicle 35 to the operation position set with respectto the product storage 21 and brings the robot 25 in sequence into thestorage starting pose for starting storage with respect to the productstorage 21, the storing poses for storing the machined workpiece grippedby the hand 29 into the product storage 21, and the storage finishingpose for finishing the storage. Thereby, the machined workpiece grippedby the hand 29 is stored into the product storage 21.

Subsequently, the automatic-operation control unit 47 moves theautomatic guided vehicle 35 to the operation position set with respectto the material storage 20 and brings the robot 25 in sequence into theextraction starting pose for staring extraction with respect to thematerial storage 20, the extracting poses for causing the hand 29 togrip an unmachined workpiece stored in the material storage 20 andextract the unmachined workpiece from the material storage 20, and theextraction finishing pose for finishing the extraction. Thereby, anunmachined workpiece is gripped by the hand 29.

Subsequently, the automatic-operation control unit 47 moves theautomatic guided vehicle 35 to the operation position set with respectto the machine tool 10 again, and causes the robot 25 to carry out theabove-described workpiece attachment, so that the unmachined workpiece Wgripped by the hand 29 is attached to the chuck 12 of the machine tool10, after which the hand 29 is moved out of the machine tool 10. In thisprocess, the automatic-operation control unit 47 causes the camera 31 tocapture an image of the identification figure when the robot 25 is inthe image capturing pose, and compensates the subsequent attachmentpreparing pose, attaching pose, and moving-away pose based on thecaptured image. Thereafter, the automatic-operation control unit 47transmits a machining start command to the machine tool 10 to cause themachine tool 10 to perform a machining operation. Note that, afterbringing the robot 25 into the attaching pose, the automatic-operationcontrol unit 47 transmits a chuck close command to the machine tool 10to close the chuck 12, so that the unmachined workpiece W is clamped bythe chuck 12.

In the system 1 according to this embodiment, unmanned and automatedproduction is continuously performed by repeating the above-describedseries of processes.

[Compensation of Operating Poses]

A manner of the above-mentioned compensation of the operating poses ofthe robot 25 in the operations carried out with respect to the machinetool 10 is described below.

<Basic Idea for Compensation>

First of all, a basic idea for the pose compensation in this embodimentis described. As shown in FIG. 5, a robot coordinate system, a cameracoordinate system, a figure coordinate system, and a goal coordinatesystem are set with respect to the robot 25, the camera 31, theidentification figure, and a targeted operating position (targetoperating position), respectively. These coordinate systems are eachdefined by three orthogonal axes x, y, and z. The origin of the robotcoordinate system is set at a freely selected point within a controlspace for the automatic-operation control unit 47 (in this embodiment,it is set at a point corresponding to the root of the robot arm). Theorigin of the camera coordinate system is, for example, set at thecenter of an image sensor of the camera 31 that is arranged in atwo-dimensional planar manner. The origin of the figure coordinatesystem is set at the center of the identification figure. The origin ofthe goal coordinate system is set at the target operating position. InFIG. 5, for the sake of expediency, only the robot coordinate system isdenoted by reference signs. In FIG. 5, the target operating position forthe robot 25 is a position in front of the chuck 12 at which the hand 29is positioned to cause the chuck 12 to clamp a workpiece W. At theposition in front of the chuck 12, the hand 29 is positioned such thatthe axis of the workpiece W is coaxial with the axis of the chuck 12.Moving the hand 29 in this state in the minus z-axis direction allowsthe chuck 12 to clamp the workpiece W.

A transformation matrix for transformation from the robot coordinatesystem to the camera coordinate system, which is calculated based oncoordinate values (x, y, z) and rotational angle values (rx, ry, rz)around coordinate axes of the camera 31 in the robot coordinate systemthat are obtained from control space information of theautomatic-operation control unit 47 and machine design data (forexample, CAD data) of the robot 25 and the camera 31, is defined asM_(camera) ^(robot). In the same manner, a transformation matrixM_(robot) ^(camera) for transformation from the camera coordinate systemto the robot coordinate system also can be calculated (obtained).

Further, a transformation matrix for transformation from the cameracoordinate system to the figure coordinate system, which is calculatedbased on coordinate values (x, y, z) and rotational angle values (rx,ry, rz) around coordinate axes of the identification figure in thecamera coordinate system that can be obtained from internal parametersof the camera 31, a homography matrix, a center coordinate, and cornercoordinates recognized from a captured image of the identificationfigure, and the size of the identification figure, is defined as M_(fig)^(camera). In the same manner, a transformation matrix fortransformation from the figure coordinate system to the cameracoordinate system is defined as M_(camera) ^(fig).

<Processing of Data Obtained in Teaching Operation>

The automatic-operation control unit 47 first obtains a transformationmatrix M_(camera_teach) ^(fig) for transformation from the figurecoordinate system to the camera coordinate system in the teachingoperation (hereinafter, “teaching-operation camera coordinate system”)and a transformation matrix M_(fig) ^(camera_teach) for transformationfrom the teaching-operation camera coordinate system to the figurecoordinate system based on the image of the identification figure(reference image) captured in the teaching operation and stored in theidentification figure image storage 45.

Subsequently, the automatic-operation control unit 47 calculates atransformation matrix M_(robot_teach) ^(goal) for transformation fromthe goal coordinate system for the hand 29 to the robot coordinatesystem in the teaching operation (hereinafter, “teaching-operation robotcoordinate system”) in accordance with a predetermined transformationequation, which is set for the purpose of control, for the operatingposes obtained in the teaching operation and stored in the operatingpose storage 43. Thereafter, the automatic-operation control unit 47calculates a transformation matrix M_(fig) ^(goal) for transformationfrom the goal coordinate system to the figure coordinate system inaccordance with Equation 1 below based on the calculated transformationmatrix M_(robot_teach) ^(goal).

M _(fig) ^(goal) =M _(fig) ^(camera_teach) ·M _(robot_teach)^(goal)  (Equation 1)

In Equation 1, M_(fig) ^(robot_teach) is a transformation matrix fortransformation from the teaching-operation robot coordinate system tothe figure coordinate system that can be calculated in accordance withEquation 2 below.

M _(fig) ^(robot_teach) =M _(fig) ^(camera_teach) ·M _(camera_teach)^(robot_teach)  (Equation 2)

Note that the transformation matrix M_(fig) ^(camera_teach) is, asdescribed above, calculated based on the image of the identificationfigure captured in the teaching operation. Further, the transformationmatrix M_(camera_teach) ^(robot_teach) is, as described above, obtainedbased on the control space information of the automatic-operationcontrol unit 47 and the machine design data (for example, CAD data) ofthe robot 25 and the camera 31.

<Compensation of Actual Operating Poses>

The automatic-operation control unit 47 first calculates (obtains) atransformation matrix M_(camera_teach) ^(fig) for transformation fromthe figure coordinate system to the camera coordinate system in actualoperation (hereinafter, “actual-operation camera coordinate system” andsometimes referred to as “current camera coordinate system”) and atransformation matrix M_(fig) ^(camera_current) for transformation fromthe actual-operation camera coordinate system to the figure coordinatesystem based on an image of the identification figure captured by thecamera 31 in the actual operation.

Subsequently, the automatic-operation control unit 47 calculates atransformation matrix M_(robot_current) ^(goal) for transformation fromthe goal coordinate system to the robot coordinate system in the actualoperation (hereinafter, “actual-operation robot coordinate system”) inaccordance with Equation 3 below based on the transformation matrixM_(fig) ^(goal) for transformation from the goal coordinate system tothe figure coordinate system that is calculated in accordance withEquation 1 above.

M _(robot_current) ^(goal) =M _(robot_current) ^(fig) ·M _(fig)^(goal)  (Equation 3)

In Equation 3, M_(robot_current) ^(fig) is a transformation matrix fortransformation from the figure coordinate system to the actual-operationrobot coordinate system that can be calculated in accordance withEquation 4 below.

M _(robot_current) ^(fig) =M _(robot_current) ^(camera_current) ·M_(camera_current) ^(fig)  (Equation 4)

Note that the transformation matrix M_(robot_current) ^(camera_current)is a transformation matrix for transformation from the actual-operationcamera coordinate system to the actual-operation robot coordinate systemthat is, as described above, obtained based on the control spaceinformation of the automatic-operation control unit 47 and the machinedesign data (for example, CAD data) of the robot 25 and the camera 31.Further, the transformation matrix M_(camera_current) ^(fig) asdescribed above, obtained based on the image of the identificationfigure captured in the actual operation.

Subsequently, the automatic-operation control unit 47 calculates atarget operating position (x_(robot_current) ^(goal), y_(robot_current)^(goal), z_(robot_current) ^(goal)) and a target operating angle(rx_(robot_current) ^(goal), ry_(robot_current) ^(goal),rz_(robot_current) ^(goal)) for the robot 29 in the actual-operationrobot coordinate system in accordance with Equations 5, 6, and 7 belowbased on the calculated (compensated) transformation matrixM_(robot_current) ^(goal) for transformation from the goal coordinatesystem to the actual-operation robot coordinate system.

$\begin{matrix}{\mspace{79mu}{M_{robot\_ current}^{goal} = \begin{bmatrix}R_{robot\_ current}^{goal} & T_{robot\_ current}^{goal} \\0 & 1\end{bmatrix}}} & \left( {{Equation}\mspace{20mu} 5} \right) \\{\mspace{79mu}{T_{robot\_ current}^{goal} = \begin{bmatrix}x_{robot\_ current}^{goal} \\y_{robot\_ current}^{goal} \\z_{robot\_ current}^{goal}\end{bmatrix}}} & \left( {{Equation}\mspace{14mu} 6} \right) \\{R_{robot\_ current}^{goal} = {{\begin{bmatrix}{\cos\;\phi} & {{- \sin}\;\phi} & 0 \\{\sin\;\phi} & {\cos\;\phi} & 0 \\0 & 0 & 1\end{bmatrix}.\begin{bmatrix}{\cos\;\theta} & 0 & {\sin\;\theta} \\0 & 1 & 0 \\{{- \sin}\;\theta} & 0 & {\cos\;\theta}\end{bmatrix}} \cdot {\quad\begin{bmatrix}1 & 0 & 0 \\0 & {\cos\;\psi} & {{- \sin}\;\psi} \\0 & {\sin\;\psi} & {\cos\;\psi}\end{bmatrix}}}} & \left( {{Equation}\mspace{14mu} 7} \right)\end{matrix}$

Note that ψ, θ, and φ in Equation 7 are rx_(robot_current) ^(goal),ry_(robot_current) ^(goal), and rz_(robot_current) ^(goal),respectively. The hand 29 is moved to the compensated target operatingposition (x_(robot_current) ^(goal), y_(robot_current) ^(goal),z_(robot_current) ^(goal)), e.g., the position indicated in FIG. 7, androtated to the target operating angle (rx_(robot_current) ^(goal),ry_(robot_current) ^(goal), rz_(robot_current) ^(goal)). The exampleshown in FIG. 7 shows a state where the hand 29 of the robot 25 ispositioned at the target operating position with no positional error,i.e., at a position in front of the chuck 12 such that the axis of theworkpiece W is coaxial with the axis of the chuck 12.

<Compensation of Image Capturing Pose in Actual Operation>

By the way, the pose for causing the camera 31 to capture an image ofthe identification figure in the teaching operation is typically setsuch that the identification figure is located at substantially thecenter of a camera frame as shown in FIG. 8, taking into account apositioning error and the like occurring in actual operation. However,if the automatic guided vehicle 35 is positioned with low accuracy inactual operation, as shown in FIG. 9, the pose of the camera 31 in theactual operation (indicated by solid lines) can differ to a great extentfrom the pose of the camera 31 in the teaching operation (indicated bydotted-and-dashed lines). Where the pose of the camera 31 in actualoperation is displaced to a great extent, it is possible that theidentification figure is shifted to an edge of the camera frame in theactual operation as shown in FIG. 10 or the camera 31 is positionedcloser to or farther from the identification figure.

Such a state leads to an unclear image of the identification figurebeing captured by the camera 31 or leads to a captured image of theidentification figure having a larger or smaller size. Such an imagecauses the transformation matrix M_(camera_current) ^(fig) obtainedbased on the identification figure to have an error, which results inthe transformation matrix M_(robot_current) ^(goal) for transformationfrom the goal coordinate system to the actual-operation robot coordinatesystem calculated based on the transformation matrix M_(robot_current)^(fig), as well as the target operating position (x_(robot_current)^(goal), y_(robot_current) ^(goal), z_(robot_current) ^(goal)) andtarget operating angle (rx_(robot_current) ^(goal), ry_(robot_current)^(goal), yz_(robot_current) ^(goal)) for the robot 29 in theactual-operation robot coordinate system, also having an error.Consequently, as shown in FIG. 11, it is not possible to position thehand 29 of the robot 25 accurately at the targeted operating position.The example shown in FIG. 11 shows a state where the hand 29 of therobot 25 is positioned at a position shifted from the targeted operatingposition, i.e., at a position in front of the chuck 12 such that theaxis of the workpiece W is shifted from the axis of the chuck 12. If thehand 29 in this state is moved in the minus z-axis direction, theworkpiece W collides with the chuck 12, so that the chuck 12 fails toclamp the workpiece W.

Accordingly, this embodiment is configured such that, in the automaticoperation carried out by the automatic-operation control unit 47, theimage capturing pose in the actual operation is compared with the imagecapturing pose in the teaching operation, and when an error of the imagecapturing pose in the actual operation is out of a predeterminedallowable range (thresholds), a process of compensating the imagecapturing pose in the actual operation is carried out so that the errorfalls within the allowable range. Note that, where the error does notfall within the allowable range after the compensation process iscarried out once, the compensation process is repeatedly carried outuntil the error falls within the allowable range.

<Checking of Image Capturing Pose>

The automatic-operation control unit 47 obtains a pair of figureposition and figure angle (x_(camera_teach) ^(fig), y_(camera_teach)^(fig), z_(camera_teach) ^(fig), rx_(camera_teach) ^(fig),ry_(camera_teach) ^(fig), rz_(camera_teach) ^(fig)) in theteaching-operation camera coordinate system by means of thetransformation matrix M_(camera_teach) ^(fig) for transformation fromthe figure coordinate system to the teaching-operation camera coordinatesystem based on the image of the identification figure captured in theteaching operation. Further, the automatic-operation control unit 47obtains the transformation matrix M_(camera_current) ^(fig) fortransformation from the figure coordinate system to the actual-operationcamera coordinate system based on an image of the identification figurecaptured in actual operation and calculates a pair of figure positionand figure angle (x_(camera_current) ^(fig), y_(camera_current) ^(fig),z_(camera_current) ^(fig), rx_(camera_current) ^(fig),ry_(camera_current) ^(fig), rz_(camera_current) ^(fig)) in theactual-operation camera coordinate system. Thereafter, theautomatic-operation control unit 47 calculates difference values (Δx,Δy, Δz, Δrx, Δry, Δrz) between the pairs of figure position and figureangle. When the difference values are out of their respectivepredetermined allowable ranges, i.e., when the following conditions arenot completely satisfied, the image capturing pose is compensated:

−0.05 mm≤Δx≤0.05 mm;−0.05 mm≤Δy≤0.05 mm;−0.05 mm≤Δz≤0.05 mm;−0.05°≤Δrx≤0.05°;−0.05°≤Δry≤0.05°; and−0.05°≤Δrz≤0.05°.Note that these thresholds for the difference values are given by way ofexample only. The thresholds for each difference value are not limitedto this example and can be empirically set as appropriate such that anaccurate image of the identification figure is captured. Further, inthis example, the image capturing pose is compensated so that all of thedifference values fall within their respective allowable ranges;however, the present invention is not limited thereto. The imagecapturing pose may be compensated so that either the difference valuesof the coordinate values (x, y, z) or the difference values of therotational angle values (rx, ry, rz) fall within their respectiveallowable ranges.

<Compensation of Image Capturing Pose>

The automatic-operation control unit 47 calculates a transformationmatrix M_(robot_current) ^(camera_teach) for transformation from theteaching-operation camera coordinate system to the actual-operationrobot coordinate system in accordance with Equation 8 below based on thetransformation matrix M_(fig) ^(camera_teach) for transformation fromthe teaching-operation camera coordinate system to the figure coordinatesystem.

M _(robot_current) ^(camera_teach) =M _(robot_current) ^(fig) ·M _(fig)^(camera_teach)  (Equation 8)

In Equation 8, M_(robot_current) ^(fig) is the transformation matrix fortransformation from the figure coordinate system to the actual-operationrobot coordinate system that can be calculated in accordance withEquation 4 above.

Subsequently, the automatic-operation control unit 47 shifts theposition of the camera 31 to a camera position (x_(robot_current)^(camera_teach), y_(robot_current) ^(camera_teach), z_(robot_current)^(camera_teach)) and a camera angle (rx_(robot_current) ^(camera_teach),ry_(robot_current) ^(camera_teach), rz_(robot_current) ^(camera_teach))calculated by means of the transformation matrix M_(robot_current)^(camera_teach) for transformation from the teaching-operation cameracoordinate system to the actual-operation robot coordinate system,thereby compensating the image capturing pose of the robot 25 (see FIG.12, where the position (pose) of the camera 31 is compensated from theposition (pose) indicated by dotted-and-dashed lines to the position(pose) indicated by solid lines). Note that this compensated cameraposition is identical to the camera position in the teaching operationin the actual-operation robot coordinate system. In ordinary cases, thiscompensation shifts the position (pose) of the camera 31 in the actualoperation to the position identical to the position (pose) of the camera31 in the teaching operation. However, since the pose of the robot 25 ischanged by this compensation, depending on the behavior of the automaticguided vehicle 35, the difference between the compensated cameraposition and the camera position in the teaching operation may not fallwithin the allowable range. In such a case, the above-describedcompensation process is repeatedly carried out until the error of theimage capturing pose in the actual operation with respect to the imagecapturing pose in the teaching operation falls within the allowablerange.

Thus, in the case where the camera position in actual operation differsfrom the camera position in the teaching operation, theautomatic-operation control unit 47 compensates the image capturing poseof the robot 25 so that the camera position in the actual operationalmost coincides with the camera position in the teaching operation, andthen causes the camera 31 to capture an image of the identificationfigure. Thereafter, the automatic-operation control unit 47 compensatesthe target operating position for the hand 29 in each subsequentoperating pose by means of Equation 3 above based on the captured image,and controls the robot 25 so that the hand 29 moves to the compensatedtarget operating position (x_(robot_current) ^(goal), y_(robot_current)^(goal), z_(robot_current) ^(goal)) and target operating angle(rx_(robot_current) ^(goal), ry_(robot_current) ^(goal),rz_(robot_current) ^(goal)). Thereby, the hand 29 is, for example asshown in FIG. 13, moved to the target operating position set in theteaching operation. Note that FIG. 13 shows a state where the hand 29 ofthe robot 25 is positioned at the target operating position with nopositional error, so that the hand 29 is positioned at a position infront of the chuck 12 such that the axis of the workpiece W is coaxialwith the axis of the chuck 12.

[General Description about Control of Operation of Robot-MountedTransportation Device with Respect to Machine Tool]

Next, control of operation of the robot-mounted transportation devicewith respect to the machine tool 10 is generally described on the basisof FIG. 14 in light of the foregoing description.

When causing the robot-mounted transportation device to carry out anoperation with respect to the machine tool 10, the automatic-operationcontrol unit 47 controls operation of the robot-mounted transportationdevice in the following manner.

That is to say, the automatic-operation control unit 47 first moves theautomatic guided vehicle 35 to the operation position set with respectto the machine tool, and then stops the automatic guided vehicle 35 atthe operation position (step S1). Subsequently, the automatic-operationcontrol unit 47 brings the robot 25 into the operation starting pose(step S2), and then brings the robot 25 into the image capturing pose(step S3). In this state, the automatic-operation control unit 47 causesthe camera 31 to capture an image of the identification figure (stepS4). Thereafter, the automatic-operation control unit 47 obtains a pairof figure position and figure angle (x_(camera_current) ^(fig),y_(camera_current) ^(fig), z_(camera_current) ^(fig),rx_(camera_current) ^(fig), ry_(camera_current) ^(fig),rz_(camera_current) ^(fig)) in the camera current coordinate system anda pair of figure position and figure angle (x_(camera_teach) ^(fig),y_(camera_teach) ^(fig), z_(camera_teach) ^(fig), rx_(camera_teach)^(fig), ry_(camera_teach) ^(fig), rz_(camera_teach) ^(fig)) in theteaching-operation camera coordinate system based on the capturedcurrent image of the identification figure and the image of theidentification figure captured in the teaching operation, and thencalculates difference values (Δx, Δy, Δz, Δrx, Δry, Δrz) between thepairs of figure position and figure angle and determines (judges)whether the difference values fall within their respective allowableranges as described above (step S5).

In the case where it is determined in step S5 that the difference valuesdo not fall within their respective allowable ranges, theautomatic-operation control unit 47 repeatedly carries out steps S4 andS5 while compensating the image capturing pose (step S6) until thedifference values fall within their respective allowable ranges.

In the case where all of the difference values fall within theirrespective allowable ranges, the automatic-operation control unit 47calculates, in accordance with Equation 3 above, pairs of targetoperating position and target operating angle (x_(robot_current)^(goal), y_(robot_current) ^(goal), z_(robot_current) ^(goal),rx_(robot_current) ^(goal), ry_(robot_current) ^(goal),rz_(robot_current) ^(goal)) that correspond to compensated operatingposes for the operating poses set in the teaching operation (step S7),and brings the robot 25 in sequence into the operating posescorresponding to the calculated pairs of target operating position andtarget operating angle (x_(robot_current) ^(goal), y_(robot_current)^(goal), z_(robot_current) ^(goal), rx_(robot_current) ^(goal),ry_(robot_current) ^(goal), rz_(robot_current) ^(goal)) (step S8).Thereafter, the automatic-operation control unit 47 brings the robot 25into the operation finishing pose (step S9) to end the operation.

As described in detail above, the system 1 according to this embodimentis configured such that, in compensating the operating poses of therobot 25, the image capturing pose of the camera 31 in the actualoperation is compensated based on an image of the identification figurecaptured by the camera 31 so that the error of the image capturing poseof the camera 31 in the actual operation with respect to the imagecapturing pose of the camera 31 in the teaching operation falls withinan allowable range; therefore, a clear and accurate image of theidentification figure is captured by the camera 31 in actual operation.Consequently, compensation of the operating poses of the robot 25 thatis carried out based on the image is carried out more accurately.

Further, the system 1 according to this embodiment is configured suchthat the identification figure arranged in the machining area of themachining tool 10 where the robot 25 actually performs the operations isused to compensate the operating poses of the robot 25; therefore, theoperating poses of the robot 25 are accurately compensated. This enablesthe robot 25 to accurately carry out even an operation which requireshigh operating accuracy.

Since the robot 25 accurately carries out operations, the system 1operates with high availability without unnecessary interruption.Consequently, the system 1 enables an unmanned system with highreliability and high production efficiency.

Further, the identification figure in this embodiment is provided on thesupport bar 15 of the tool presetter 13 that is stored outside themachining area while machining is performed by the machine tool 10;therefore, the identification figure is prevented from being soiled bychips or the like produced during machining. Consequently, thecompensation is carried out accurately.

Further, the identification figure in this embodiment has a matrixstructure having a plurality of pixels arranged two-dimensionally;therefore, the operating poses are compensated with high accuracy.

Hereinbefore, one embodiment of the present invention has beendescribed. However, it should be understood that the present inventionis not limited to the above-described embodiment and can be implementedin different manners.

For example, in the above-described embodiment, the identificationfigure has a matrix structure having a plurality of pixels arrangedtwo-dimensionally. However, the identification figure is not limited tosuch a figure and may be any other suitable figure which allows forcompensation of the pose of the robot 25 based on a captured image ofthe figure.

Further, in the above-described embodiment, the operating poses of therobot 25 in actual operation are compensated based on a captured imageof the identification figure without calculating an amount of error ofthe operating poses of the robot 25 in the actual operation with respectto the pose of the robot 25 in the teaching operation. However, thepresent invention is not limited to such a configuration and theoperating poses of the robot 25 in actual operation may be compensatedby any other suitable method. For example, a configuration is possiblein which an amount of error of the image capturing pose of the robot 25for capturing of an image of the identification figure in actualoperation with respect to the image capturing pose of the robot 25 inthe teaching operation is calculated and the other operating poses ofthe robot 25 are compensated based on the calculated amount of error.

Further, in the above-described embodiment, the operating poses of therobot 25 are compensated by compensating an error thereof in athree-dimensional space. However, the present invention is not limitedto such a configuration and the operating poses of the robot 25 may becompensated by compensating an error thereof in a plane defined by twoparticular orthogonal axes. For example, a configuration is possible inwhich the identification figure is arranged horizontally and positionalerrors in a first axis and a second axis in a plane including theidentification figure and a rotational error around an axisperpendicular to the plane are compensated.

Further, in the above-described embodiment, an example configuration isdescribed in which the automatic guided vehicle 35 is used. However, thepresent invention is not limited to such a configuration and theautomatic guided vehicle 35 may be replaced with a transportation devicewhich is able to be moved by a human operator pushing it, such as agenerally used carriage. In such a case, a configuration is possible inwhich the robot 25 is mounted on the transportation device and thetransportation device is moved to the operation position set withrespect to the machine tool 10 by human power to cause the robot 25 tocarry out attachment or removal of a workpiece to or from the machinetool 10.

Further, in the above-described embodiment, the display board 16, i.e.,the identification figure, is arranged horizontally in the machine tool10. However, the present invention is not limited to such aconfiguration and the identification figure may be arranged in parallelwith a vertical plane.

Further, in the above-described embodiment, a vertical lathe as anexample machine tool is described. However, the present invention is notlimited to application to a vertical lathe and is applicable also toother known types of machine tools, such as a horizontal lathe, avertical machining center, a horizontal machining center, and a combinedmachine tool having a tool spindle and a workpiece spindle.

For example, in the case of a horizontal lathe 100 as illustrated inFIG. 15 that has a tool spindle 105 rotating a tool, the lathe 100 canbe configured such that, as illustrated in FIG. 15, the display board 16is supported horizontally by a holder 106 and the holder 106 is to beattached to the tool spindle 105. Alternatively, a configuration ispossible in which, as illustrated in FIG. 16, a display board 112 issupported vertically by a holder 111. In each case, the holder 106, 111is stored in a tool magazine as a tool storage while machining isperformed by the lathe 100, 110, and the holder 106, 111 is extractedfrom the tool magazine and attached to the tool spindle 105 when therobot 25 performs an operation. Note that, in FIGS. 15 and 16, referencenumeral 101 denotes a first spindle, reference numeral 103 denotes asecond spindle, and these spindles are arranged to be coaxial with eachother and face each other. Further, reference numeral 102 denotes afirst chuck attached to the first spindle 101 and reference numeral 104denotes a second chuck attached to the second spindle 103. Further,reference numeral 107 denotes a tool rest, reference numeral 108 denotesa turret mounted on the tool rest 107, and reference numeral 109 denotesa support jig attached to an outer surface of the turret 108 to supporta workpiece W″.

Further, in the above-described embodiment, the robot coordinate system,the camera coordinate system, the figure coordinate system, and the goalcoordinate system each have the x-axis and the y-axis set in ahorizontal plane and the z-axis set vertically. However, the presentinvention is not limited to such a configuration. The directions of thecoordinate axes can be set freely.

Further, the above-described embodiment is described mainly with respectto an example in which the robot 25 attaches and removes a workpiece W(W′). However, the present disclosure is not limited thereto. The targetobject to be handled by the robot 25 may be any other suitable objectwhich is able to be attached to and removed from the machine tool 10,examples of which include, besides the above-mentioned workpiece W (W′),a tool, an ATC camera, and a measurement device.

Below is a possible variation of the present invention.

(Variation)

The above-described embodiment is described with respect to an examplein which an image including the identification figure is captured by thecamera and the position of the camera is calculated based on theposition of the identification figure in the image to control movementof the camera to a preset camera position. However, the presentinvention is not limited to this example.

This variation is configured not to use the identification figure. Inthis variation, a profile of an internal shape of the machine tool in atwo-dimensional image is detected from a captured image and the detectedprofile is compared with preset CAD data on shape. The ratio of matchingpoints between profile data extracted from the image and two-dimensionalshape data generated based on the CAD data is evaluated to identify ashape having a profile having many matching points (a high degree ofmatching) with the preset shape, and the position of the camera iscalculated based on the position of the identified shape. Because of notusing the identification figure, this variation is able to use the shapeof an internal structure of the machine tool as reference. Examples ofthe internal structure include a chuck, a tool, a spindle, a turret, atool presetter, a table, and a pallet. The internal structure may be, ofcourse, the identification figure.

In three-dimensional recognition based on internal structure profile(shape), judgment may be made by evaluating the number of matchingpoints between an edge extracted from an image and a profile obtained byprojecting a three-dimensional model created based on three-dimensionalCAD data or the like into a two-dimensional image. In three-dimensionalrecognition based on three-dimensional point cloud, judgment may be madeon the basis of evaluation based on the number of matching pointsbetween a three-dimensional point cloud measured by a predeterminedmethod and a three-dimensional model.

This variation can provide the following image processing, namely,

an image processing method of processing an image captured by a cameraof a robot-mounted transportation device, the robot-mountedtransportation device including: (i) the camera capturing the image;(ii) a robot having the camera attached thereto and having an actingpart acting on a target object; (iii) a moving device having the robotmounted thereon and configured to be movable; and (iv) a controllercontrolling a position of the acting part,

the image processing method including:

a first calculation step of analyzing a first image of an internalstructure of a machine tool captured by the camera and calculating acurrent position of the camera based on analysis of the internalstructure in the first image;

a first control step of controlling motion of the robot to move thecamera from the current position of the camera to a preset position ofthe camera;

a second calculation step of, after moving the camera, causing thecamera to capture an image of the internal structure of the machinetool, analyzing the internal structure in the captured second image, andcalculating a current position of the acting part; and

a second control step of controlling motion of the robot to move theacting part from the current position of the acting part to a presetposition of the target object or a preset position to which the targetobject is to be transported.

This variation can also provide the following image processingapparatus, namely,

an image processing apparatus processing an image captured by a cameraof a robot-mounted transportation device, the robot-mountedtransportation device including: (i) the camera capturing the image;(ii) a robot having the camera attached thereto and having an actingpart acting on a target object; (iii) a moving device having the robotmounted thereon and configured to be movable; and (iv) a controllercontrolling a position of the acting part,

the image processing apparatus including:

a first calculator configured to analyze a first image of an internalstructure of a machine tool captured by the camera and calculate acurrent position of the camera based on analysis of the internalstructure in the first image;

a first control unit configured to control motion of the robot to movethe camera from the current position of the camera to a preset positionof the camera;

a second calculator configured to, after moving the camera, cause thecamera to capture an image of the internal structure of the machinetool, analyze the internal structure in the captured second image, andcalculate a current position of the acting part; and

a second control unit configured to control motion of the robot to movethe acting part from the current position of the acting part to a presetposition of the target object or a preset position to which the targetobject is to be transported.

In this image processing, an internal structure of the machine tool isused instead of the identification figure to identify the position ofthe camera. Note that the other configurations and control processesthat are usable in the image processing using a captured image of theidentification figure are, of course, applicable to this variation.

As already mentioned above, the above description of the embodiments isnot limiting but illustrative in all aspects. One skilled in the artwould be able to make variations and modifications as appropriate. Thescope of the present invention is not defined by the above-describedembodiments, but is defined by the appended claims. Further, the scopeof the present invention encompasses all modifications made within thescope equivalent to the scope of the claims.

REFERENCE SIGNS LIST

-   -   1 System    -   10 Machine tool    -   11 Spindle    -   12 Chuck    -   13 Tool presetter    -   14 Contactor    -   15 Support bar    -   16 Display board    -   20 Material storage    -   21 Product storage    -   25 Robot    -   29 Hand    -   31 Camera    -   35 Automatic guided vehicle    -   37 Operation panel    -   40 Controller    -   41 Operation program storage    -   42 Moving position storage    -   43 Operating pose storage    -   44 Map information storage    -   45 Identification figure image storage    -   46 Manual-operation control unit    -   47 Automatic-operation control unit    -   48 Map information generator    -   49 Position recognition unit    -   50 Input and output interface    -   W Unmachined workpiece    -   W′ Machined workpiece

What is claimed is:
 1. An image processing method of processing an imagecaptured by a camera of a robot-mounted transportation device, therobot-mounted transportation device including: (i) the camera capturingthe image; (ii) a robot having the camera attached thereto and having anacting part acting on a target object; (iii) a moving device having therobot mounted thereon and configured to be movable; and (iv) acontroller controlling a position of the acting part, the imageprocessing method comprising: a first calculation step of analyzing afirst image of an internal structure of a machine tool captured by thecamera and calculating a current position of the camera based onanalysis of the internal structure in the first image; a first controlstep of controlling motion of the robot to move the camera from thecurrent position of the camera to a preset position of the camera; asecond calculation step of, after moving the camera, causing the camerato capture an image of the internal structure of the machine tool,analyzing the internal structure in the captured second image, andcalculating a current position of the acting part; and a second controlstep of controlling motion of the robot to move the acting part from thecurrent position of the acting part to a preset position of the targetobject or a preset position to which the target object is to betransported.
 2. The image processing method according to claim 1,wherein the internal structure is an identification figure, a chuck, atool, a spindle, a turret, a tool presetter, a table, or a pallet.
 3. Animage processing apparatus processing an image captured by a camera of arobot-mounted transportation device, the robot-mounted transportationdevice including: (i) the camera capturing the image; (ii) a robothaving the camera attached thereto and having an acting part acting on atarget object; (iii) a moving device having the robot mounted thereonand configured to be movable; and (iv) a controller controlling aposition of the acting part, the image processing apparatus comprising:a first calculator configured to analyze a first image of an internalstructure of a machine tool captured by the camera and calculate acurrent position of the camera based on analysis of the internalstructure in the first image; a first control unit configured to controlmotion of the robot to move the camera from the current position of thecamera to a preset position of the camera; a second calculatorconfigured to, after moving the camera, cause the camera to capture animage of the internal structure of the machine tool, analyze theinternal structure in the captured second image, and calculate a currentposition of the acting part; and a second control unit configured tocontrol motion of the robot to move the acting part from the currentposition of the acting part to a preset position of the target object ora preset position to which the target object is to be transported.
 4. Arobot-mounted transportation device comprising: a robot having a camerafor image capturing and having an acting part acting on an operationtarget; a transportation device having the robot mounted thereon andconfigured to be movable to an operation position set with respect tothe operation target; and a controller configured to, in accordance withan operation program containing a predetermined operation command, bringthe robot in sequence into an operation starting pose, an imagecapturing pose for causing the camera to capture an image of anidentification figure for pose compensation provided on the operationtarget, and at least one operating pose for causing the acting part toact on the operation target, the operation starting pose, the imagecapturing image, and the operating pose being set in advance byperforming a teaching operation with respect to the robot, theidentification figure being formed on a predetermined plane and arrangedon the operation target, the controller being configured to carry out: aprocess of obtaining teaching-operation positional relation informationindicative of positional relation between the identification figure andthe camera in the teaching operation based on an image of theidentification figure captured by the camera with the robot in the imagecapturing pose in the teaching operation; a process of, in actualoperation of the robot in accordance with the operation program,obtaining actual-operation positional relation information indicative ofpositional relation between the identification figure and the camera inthe actual operation based on an image of the identification figurecaptured by the camera after the transportation device is moved to theoperation position and the robot is brought into the image capturingpose from the operation starting pose in the actual operation; a processof comparing the obtained actual-operation positional relationinformation with the obtained teaching-operation positional relationinformation to determine whether a difference value between them iswithin a predetermined allowable range; and when the difference value isnot within the allowable range, a process of compensating an imagecapturing pose of the camera with respect to the identification figureso that the difference value falls within the allowable range, thecontroller being further configured to, when the difference valuebetween the actual-operation positional relation information and theteaching-operation positional relation information is within theallowable range, operate the robot such that the robot is brought into acompensated operating pose obtained by compensating the operating poseset in the teaching operation based on a most recently captured image ofthe identification figure.
 5. A system comprising: a machine toolperforming predetermined machining on a workpiece; a robot having acamera for image capturing, having an acting part acting on theworkpiece, and performing an operation with respect to the machine tool;a transportation device having the robot mounted thereon and configuredto be movable to an operation position set with respect to the machinetool; and a controller configured to, in accordance with an operationprogram containing a predetermined operation command, bring the robot insequence into an operation starting pose, an image capturing pose forcausing the camera to capture an image of an identification figure forpose compensation provided on the machine tool, and at least oneoperating pose for causing the acting part to act on the workpiece, theoperation starting pose, the image capturing image, and the operatingpose being set in advance by performing a teaching operation withrespect to the robot, the identification figure being formed on apredetermined plane and arranged in a machining area of the machinetool, the controller being configured to carry out: a process ofobtaining teaching-operation positional relation information indicativeof positional relation between the identification figure and the camerain the teaching operation based on an image of the identification figurecaptured by the camera with the robot in the image capturing pose in theteaching operation; a process of, in actual operation of the robot inaccordance with the operation program, obtaining actual-operationpositional relation information indicative of positional relationbetween the identification figure and the camera in the actual operationbased on an image of the identification figure captured by the cameraafter the transportation device is moved to the operation position andthe robot is brought into the image capturing pose from the operationstarting pose in the actual operation; a process of comparing theobtained actual-operation positional relation information with theobtained teaching-operation positional relation information to determinewhether a difference value between them is within a predeterminedallowable range; and when the difference value is not within theallowable range, a process of compensating an image capturing pose ofthe camera with respect to the identification figure so that thedifference value falls within the allowable range, the controller beingfurther configured to, when the difference value between theactual-operation positional relation information and theteaching-operation positional relation information is within theallowable range, operate the robot such that the robot is brought into acompensated operating pose obtained by compensating the operating poseset in the teaching operation based on a most recently captured image ofthe identification figure.
 6. The system according to claim 5, whereinthe controller is configured to repeatedly carry out the process ofcompensating the image capturing pose of the camera with respect to theidentification figure two or more times until the difference value fallswithin the allowable range.
 7. The system according to claim 5, whereinthe transportation device is an automatic guided vehicle controlled bythe controller and is configured to move to the operation position setwith respect to the machine tool under control by the controller.
 8. Thesystem according to claim 5, wherein the identification figure has amatrix structure having a plurality of pixels arrangedtwo-dimensionally.